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Abstract 

We construct a sequential adaptive procedure for estimating the autoregressive 
function at a given point in nonparametric autoregression models with Gaussian 
noise. We make use of the sequential kernel estimators. The optimal adaptive con- 
vergence rate is given as well as the upper bound for the minimax risk. 
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1 Introduction 

Our problem is the following. Suppose we observe data from the model : 



l<k<n. 



where Xk = k/n and {C,k)ke{i,...,n} are random variables independent and identically dis- 
tributed by standard Gaussian. 

The mode l f|l.ip is a generalization of an autoregressive processes of the first order. 



In 



Dahlhaus 



(Il996al ). the process fll.lD is considered w ith the function S, having a para- 



metric form. Moreover, the paper of iDahlhaus I (Il996bl ) studi es spectra l proper ties of the 



stationary process (11. ip with the nonparametric function S. 



Belitser 



(l2000al ) considers 
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the model (II. II) with Lipschitz conditions and proposes a recursive estimator. The author 
estabhshes the convergence rate for quadratic risk. 

This paper deals with a nonparametric estimation of the autoregressive function S at 
a fixed point zq e]0; 1[, when the smoothness of S is unknown. More precisely, we assume 
that the function S belongs to a Holder class with unknown regularit y < /3 < 1. Not e 



Belitser 



J2000al ). 



that for (3 = 1 this gives the class of Lipschitz functions, considered in 
The goal of this paper is to find an adaptive minimax convergence rate and to construct 
an adaptive estimate. 

Many studies is devoted to the minimax convergence rate or an asymptotically efficient 
estimator in adaptive non sequential setting, i.e. when one or more parameters of the 
model are assumed to be unknown, in particular , the regularity of the function. The 
first result in this direction is obtained in Ihepskii's I ( 1990 ). where the author proposed an 
adaptive pointwise estimation method for the Gaussian white noise model. He constructed 
an adaptive estimation p rocedure which is minimax for functions f rom the Holder classes 
with unknown regularity. iGaltchouk and Pergamenshchikovl ( 120011 ) modified the Lepskii's 
method for the sequential a daptive estimation for the drift of the diff usion processes. 

In this paper, similarly to IGaltchouk and Pergamenshchikovl ( 1200 ll ). we apply the Lep- 
skii procedure to the model ( II. ip based on the sequential ker nel estimates. We construct 



Borisov and Konev 



(Il97 



the sequential kernel estimator using the method proposed in 
for the parametric case. It should be noted that to apply the Lepskii procedure the kernel 
estimators must to have the distribution tail of the Gaussian type. To obtain this property 
one needs to use the sequential approach. To this end we show some modification of the 
Levy theorem for discrete time and then, using this result, we show that the sequential 
kernel estimators have the the same form for the distribution tail as a Gaussian ran- 
dom variable. It should be noted that non- sequential kernel estimation does not have the 
above property in the case of the model (11.11) . Thus, in this case, the adaptive pointwise 
estimation is possible only in the sequential framework. 

Let we describe now the sequential kernel estimators. For a constant H > 0, we define 
a^, < a^j < 1, such that 



i=i 



where the kernel Q{-) is the indicator function on the interval [—1; 1], and t^j is the 
stopping time defined as follows: 



Th = inf{l <k<n:J2 Qi^j) v]-i > H}- 



1.2) 
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Note that 

k 

Thus the kernel estimator is written as follows: 

Such an estimator is very convenient to calculate the quantity Fi\S^ ^ (zq) — S{zq)\. 

We describe in detail the statement of the problem in section 2. In section 3 we prove 
the result of an asymptotic lower bound of adaptive minimax risk. Section 4 is devoted 
to proving the asymptotic upper bound for the risk of the kernel estimator f ll.3p . Section 
5 gives the appendix which contains some technical results. Finally, we illustrate the 
obtained results by numerical examples. 



2 Statement of the problem 

The problem is to estimate the function S" at a fixed point Zq g]0, 1[, i.e. the value S{zq). 
For any estimate Sn = S^{zo) (i.e. any measurable with respect to the observations 
{yk)i<k<n function), the risk is defined on the neighborhood Ti^^^z^, K,e) by 

7^„(^0= sup sup N{f3)Es\S^{zo)-S{zo)\, (2.1) 

/ n \/3/(2/3+l) 

where N[I3) = ) corresponds to the convergence rate of adaptive estimators 

Vln n/ 

on class T-L^^^Zq, K,e) and is the expectation taken with respect to the distribution 
of the vector {yi, ...,?/„) in (11. ip corresponding to the function S. 
We consider model (11.11) where S G C]^([0, 1],M) is the unknown function. To obtain 
the stable (uniformly with respect to the function S ) model (II. ID . we assume that for 
some fixed < e < 1, the unknown function 5* belongs to the stability set 

r, = {SGCi(]0,l],M):||S||<l-£}, (2.2) 

where 115*11 = supo<2:<i |5'(x)|. Here C|]0, 1] is the Banach space of continuously differ- 
entiable ]0, 1] — t- M functions. For fixed constants K > and < /3 < 1, we define the 
corresponding stable local Holder class at the point 

H^P\z^,K,e) = {SeV^: n*{z^,S) < K} , (2.3) 

with 

n*{z,,S)= sup l^(^)-^(^o)l 



xe[o,i] 



X Zrt 
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The regularity f3 G is supposed to be unknown, where the interval is 

known. 

First we give the lower bound for the minimax risk. We show that with the convergence 
rate N{P) the lower bound for the minimax risk is strictly positive. 

Theorem 2.1. The risk (12. ip admits the following lower bound: 

liminf inf TZn{Sn) > 7 , 

n— >-oo 4 

where the infimum is taken over all estimators Sn- 

Now we give the upper bound for the minimax risk of the sequential adaptive estimator 
defined in (11. 3p . Since /3 is unknown, one can not use this estimator because the bandwidth 
hn depends on /3. That is why we partition the interval [(3^; (3*] to follow a procedure of 
Lepskii. Let us set 

1 

1 \ 2/3+1 



d^ = n/\nn and h{l3) = j . (2.4) 
We define the grid on the interval [/3^; (3*] with the points : 

/3fc = /3. + -(/3*-/3J, A; = 0,...,m with m = [\ndj + l. (2.5) 
m 

We denote N;., hj^, and oj{hj) as 
and 

uj{h-) = max i \S* - St | - 



^ o<k<j V Ni^+i 
We also define the optimal index of the bandwidth as 

k = mil^O<j<m:co{hj)>^^-l. (2.6) 
We note that uj{h^ = —X/N-^ and thus k > 0. The positive parameter. A, is chosen as 

X> K + 6^14:+ ^ 



2/3, + 1 

The adaptive estimator is now defined as 

^„ = 5^_^ with h = h^^. (2.7) 

The following result gives the upper bound for the minimax risk of the sequential adaptive 
estimator defined above. 
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Theorem 2.2. For all < e < 1, we have 



hmsup 7^,„(^„) < cx). (2.8) 

n— >oo 

Remark 2.3. Theorem \2. 1\ gives the lower hound for the adaptive risk, i.e. the convergence 
rate N{P) is best for the adapted risk. Moreover, by Theorem \2.2\ the adaptive estimates 
(12. 7p possesses this convergence rate. In this case, this estimates is called optimal in sense 
of the adaptive risk (12.11) 



3 The lower bound 

We show that with this appropriate rate, N{I3), the lower bound of minimax risk is strictly 
positive. 

Proof of Theorem 12.11 

To simplify notations, we denote A^(/3*) = A^*, N{(3*) = N* and h{f3^) = h^. 
We choose 5* as 

where V is a. function of C°° class with compact support [—1,1] such that 

/ V [u)du = — with p = — — -— — , 

J-i ' 2 (2/3* + 1) (2/3, + 1) ' 

and satisfying V{fS) = 1 and V{u) = for \u\ > 1. 

It is easy to show that for all real K, large enough, S G T-L^^'^Zq, K,e). Note that for 
all S, the measure P5 is equivalent to the measure Pq, where Pq is the distribution of 
vector (yi, . . . ,?/„) in (11.11) corresponding to function Sq = 0. It is also clear that in this 
case, the density of Radon-Nikodym can be written as 



dPo. . 
■ = T5-(2/i,---,2/n) 
at^s 



1_2 



with 
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We define 

t{S) = 1-S\zo). (3.1) 



According to Lemma I5.2[ we obtain 



Ps- lim ^,l = Ps- lim i^i^^' ( 

\ fc=l ^ 



-J- 1 fc- 



1 /"\^2/^a;-Zo 



Ps - lim — - / l^M — 1 dx 



1 

V'^{u)du =- = ^, 



1 



/3 ^2 
2 



since r(^) = 1 - 

* 

Furthermore, using a central limit theorem for martingales (cf. Lemma [5 .Op . it is easy 
to see that under the measure P^, 

rjn =^ A/'(0, 1) when n ^ oo . 

In fact, we can rewrite //„ as follows : 




" ^'^ k=i 



with 

Uk,n = ^= V — yk-1 

V li* \ n* / 

Let us consider the first condition of lemma 15.61 To verify this, it suffices to show that 

n 

Z2 ^si'^l,n'^{\uk.^\>e)\J^k-l,n) > 0. 

k=l 

We have 



n 



X] ^si'^k,n'^{\uk,,A>e)\J^k-l,n) = ^ E_5 (Wfc,„l(|«fe,„|>e)) (3.2) 



A;=l k=l 

k=k* , X 

X — ^ 2 / — \ / 2 2 

(^^^jE,(y,_,41(K,j>.)J, 



where 

= [nzo — nhn] + 1 and k* = [uzq + nhn] , (3.3) 
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with 

Es(?/Li4'l(K„|>.)) <jEsytiEsekJPs{M>e) 



<\/^syt-i^s^t\l-2^s<n 



where Ci and C2 are constants independent of n. So the term in (13. 2 p is bounded above 
by 

E5 E E5(«',nl(|..,J>.)l-^fc-l,n) < -f E ' (3.4) 

where C3 is a new constant and as n — )• 00, fl3.4[) tends to zero. 
The second condition is easily verified 



fc=i ^* ^ fc=i V / 



rTT E ^ ( ) ^^-1 



Let us denote On = A^=k|S'„|. We have 

n^iSn) > max(Es,N*\SJ,EsN,\Sn- Sizo)\ 

N* 

iv; 



' N* (iT* 

>^E, (— (3.5) 



We set 7„ = — — . We can rewrite (13. 5 p as: 

7^J^n) > ^Es{-fnPn\On\+ |l-^n|). 

Let -B„ = {rjn < 0} and C„ = < /?}. Clearly, when i?„ fl C„ is realized, we have 

— ^ n 

7nPn > exp{/3 lnd„ - 2^}- 



The right-hand side of this inequahty tends to oo as n approach oo. This means that for 
n sufficiently large, 



Since 



and 



hence 



> ^E^ls^ncjTnPn l^nl + |1 - ^n|) 
>iE5lB„nC„(|^n|+ 1-l^nl) 

= ^P5(i?„na). (3.6) 



— > 

n n^oo z 



Ps{Cl) > 0. 



As PJBn) = 1/2, we deduce that P JBr, n C„) y 1/2. 

n— s>oo 

Passing to the limit as n — > oo in (13.61) . we obtain the desired result. □ 

4 Sequential adaptive estimation (upper bound) 

Proof of Theorem [272] 

We proceed by following a method based on sequential analysis. First, we rewrite the 
estimation error as follows: 

(4.1) 



where 

H 



and 
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Note that the first term in the right-hand side term of (14. ip is studied in Lemma 15.31 We 
can show directly that for every S E 1-L^^\zq^ K, e) 



\BH{h)\<Kh^ (4.2) 



and also, using Lemma [5.51 we have 



sup sup E5|C^(/i)| < oo, (4.3) 

n>l h^<h<h* 

where = h{(3J and h* = h{(3*). Now, we choose H = nh and 

L = M{k > : > /3} - 1 . 

This means 

A</3<A+i and K<hiP)<h^^,. 
In the sequel, we denote S^{zo) = S^f^{zo). We have now 

\Sliz,) - S{Z,)\ < \AJH.)<nH.) + + IG(/^JI 

and 

Inequality (14. 3 p implies 

limsup sup A^(/3) sup Y^gw{i,z^ < oo , (4.4) 

where 

w{i,z,) = \SlJz,) - S{z,)\ + \Sliz,) - S{z,)\ . 
Now considering the estimator 5^, one has 

l^„(^o)-^(^o)l< A + ^2 + ^^^o), (4-5) 

where 

A = \Sn{zo) - 5(zo)|l|fc>,+i} and = |5„(zo) - 5(zo)|l^^<^_2} • 
We focus now on the left-hand side in this inequality. We have 

\Sn{zo) - 5(zo)|l{fe>,+i} < |5|(zo) - 5;jzo)|l{fe>.+i} + l^li^o) - '5(^o)|l{fc>.+i} • 
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Moreover, 



This implies directly that 

limsup sup N{f3) sup < oo . (4.6) 

We establish now a bound for the right-hand side of fl4.5p : 



{A;<t-2} ' 



where 



r= max \Cni^)\- (4-7) 

l<j<m ■' 



Note that 

{^<^-2} = U {^(^•)>A/iVj . 

Moreover, 



1=0 

^ U ({l^;.(^o) - Siz,)\ > \/N^} u {|s;(^o) - 5(^o)l > AM+J 



i=0 

i-i 

U 

also for j < 6 — 1, 
For Z < 6 - 1 

A^, , 1 (h,)" < exp{- , ^^"^"^ } < 1, 

and 

> exp{ -} = e . 



^i+i ^ 



In the first set on the right-hand side (14.81) , by Lemma 15.21 we prove that for n sufficiently 

4 

large and for A > + /4 + — -, we have 
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c ||C„(MI > - ) } 

We also have (l/dn)^^'-'^^'^^'' y/nh = ^/n/dn so that the last inclusion becomes 



{\Sl\z,) - S{z,)\ > X/N^} C \Uh,)\ >{X-K)J-\. 



n 




Similarly for the second set on the right-hand side in f l4.8p . we obtain 

{\Sl^{z,) - S{z,)\ > X/N,^,} C ||C„(/.OI > (A - K)/e ^^j . 

Finally, 

{:fc<^-2}c{c>Ai v^}, 

with = (A — K)/e. So one has 

K 1 

Using Lemma [5.21 for t > 2, one can easily estimate the first term on the right-hand 
side of inequality (14. 9p by 



PsiMh) < ^h) = J2^siMhi) < nhu k = l) 

1=1 

m 

< Y^PsiMhi) <nhi) 
1=1 

m / 1 /•! 



= gP^ (^A„,(Q, < 1 - 

m 

<5^P^(|A„(Q,/.0I>1) 

m 

< A^*(Q, /^O < ([lnrf„] + 1) C, R^' {h*f'P. 

1=1 
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Consider now the last term in the right-hand side of inequahty (14. 9p . We have 

r+oo 
J 

= / P5(C>;^,r> Aiv^)rfz 

= \ -J^Vs^C > Ai ThiTI) + / Pc;(C* > z) dz. 

Using (14. 7p and Lemma [5 .Sj we have 

Ps(C > ^) = P5( max |Cn(^-)l>^) 

m 

= Y.^si\Uh,\)>z) 

i=i 



Then, 



+00 



'In n 
f +00 



< 2mAi v^e-5^?^''" + 2m / ze-^'^^dz 

< (^Ai A/hin + 4^ 2m n"^?/^ 
which imphes inequahty f l2.8p . □ 

5 Appendix 

In this section, we study the properties of stationary processes in the model fll.ip . 

Lemma 5.1. For all t E N* and < £ < 1, the random variables in (11. ip satisfy the 
following : 

r* = sup sup sup ^svf ^ (^•■^) 

n>i o<fc<n 5er^ 

Proof. 

Assume that ?/o = 0. Model f ll.ip becomes 

k k 

1=1 i=i+i 
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with S G and for all 1 < A; < n, 



2t 



Moreover, the Holder inequality, with p = 2t , gives 



Thus, it follows that 



s^k — 2*t! 
and we get the desired result. □ 

Let us introduce the following notation : 

1 " 1 

Lemma 5.2. Let f be a function twice continuously differentiahle in [—1,1], such that 
f{u) = for \u\ > 1. Then for all t e W, 

limsup sup sup— ^ sup sup Eg A^*(/, /i) < , (5.2) 

n^oo h,<h<h'' R>0 ri IL ' \\f\\^<R S&iP[zo,K,e) 

^/^ere||/l|i = 11/11 + 11/11 and C, = 2^'K'\r*f. 
Proof. First, write 

n 

^ fiuk)yl^i = T^ + a„, (5.3) 
fc=i 

where 

k* k* 

'^n=Yl f^^k)yl and a„ = ^ (/(mJ - /(m^.J) yl_^ - fiu,^*) yl, , 

fc=fc* k=k^ 

for the integers k* and fc^ defined as in (13. 3p . Substituting into model (II. ip gives us 

k" 
k=k, 
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where 



k* 



Lif) = fi^k) and M„ = ^ f{uk) {2 S{xk) yk-i^k + Vk) 



with ?7fc = ^1 — 1. Noting that 



Cn=Yl i^^i^k) - S'^izo)) f{uk) yl_i and Dn = f{uk){yl_i - yl) 



k=k. 



we obtain 



nh " 



1 W) , 1 K 



t{S) nh r(5') nh 



with H^ = M^ + C^ + S^{zo) D^. Moreover, it is easy to see that 



W) 

nh 



f{uk)dt 



f{t)dt 



k=k, "'"fe-l 

Recall also that II f II + II f II < R. Then 



1 /"^ 



< 



R 

nh 



The definition in (13. ip implies that for any 5 G F^, 

< r(S) < 1. 



(5.4) 



(5.5) 



Taking into account (15. 3p and the lower bound for t{S) given in (15.50 . we prove that 



nh t{S) J _i 



f{t)dt 



< 



R IVL C„ D,, 



n I n 



\ nh nh nh nh 



(5.6) 



We note that is the last term of the square integrable martingale (Gj) j<k*, where 



Gj = ^ f{uk) {2S{xk)yk~iik + rjk) 



k=kt 



So, by applying the Burkholder inequality, it comes 



nh 



2t j^2t 



M„ < 



k* 



2t 



{nh) 



2t 



E5 X] ^^(^fc) {'^S{xk)yk-iC.k + VkY 



\k=k. 



R' 



k* 



2t 



< 



2* {nhy+ 
R' 



k=k. 



24t-2 ^2t 



{nhy 



2* \ 2H\ 



2r* + 



m 

2H\ 



+ 1 
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where A^^ = 18{2tf^^/{2t - 1)^/2 ^nd r* is given in (15. ip . Since, \S{xk) — S{zo)\ < 
K\xk — zq\I^ for all S G l-i^ {zq, K,e) and after applying the Holder inequality for p = 2t 
and g = 2t/(2t — 1), we obtain 



(A;* \ fc* 



{nhf' ^ " - (nh) 

Now, consider the last term in the right-hand side of inequality fl5.4p . can be written 

as 

k* 
k=k. 

Since 11/11 + 11/11 < i?, we have 



\ k=kt / 

Similarly we find a bound for the second term in the right-hand side of the expression 
(15. 2p . Hence we have Lemma 15. 2[ □ 

Lemma 5.3. For any t>l, the stopping time defined in (11. 2p satisfies the following 
property: for H = nh 

Ps{rH>n)<C,{Rhf'^, 

where is defined in (15. 2p . 
Proof. 

Taking into account that r{S) < 1, we obtain 

1 " H 
PsiTH >n)= P^(— QM VU < ^) 



k=l 



1 

— — / Q{u)du + A„(g, h) < 1 
J-i 



Ps(An{Q,h)<l ^ 



r{S), 

< Ps (I A„(Q, h)\>l)<Es Al\Q, h) < C, R'' h''^ . 



This last inequality comes from Lemma 15.21 □ 



To prove Lemma 15.51 we need the following lemma proved in iLiptser and Shiryaev 



(119781) p.234-235. 
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Lemma 5.4. Let the Wiener process W = {Wt,J-'t),t > 0, be given on a probability space 
and let there also be given the random process f = {ft,J^t), ^ > 0, such that : 



T 

2 



(1) P / ftdt <oo =l, 0<T<oo, 



oo 



(2) //cit = ooJ=l. 

Then the random process z — (z^jF^), s > 0, with Zg — J^" ftdWt, = J>^, where 
Tg — inf : Jg fudu > s), is a Wiener process and with probability one 

lim = 0. 

*-°o jlfldu 

Lemma 5.5. For all z >2 and H > 0, one has 

Psi\CH{h)\>z)<2e-^'/\ (5.7) 

Proof. The Brownian motion, (Wt)t>o, is a stochastic process whose disjoint increments 
are independent as Wt+s—Wt, follows a Gaussian distribution with zero mean and variance 
s. So in our case, we can write 

We recall that 
Thus, 



where 



and 



k=l 



r„-l 



k=l 

with Sk = Q{uk) Vk-i l(fc<fc*) + l(fc>r) and 

k 



Th = ini{k > 1 : E - 
16 



One can see that 



where 



with 



Indeed, 



We set 



with 



Psi\CHih)\>z) = P, 



> z 



6'. 



J > fn- 



I ft^'^t = ^ I ft = s'i K ~ 



6" 

J 



6, 


j 


< 






= 





j 


> 



which yields, 



Jo Jj-1 



T S' + a, 5f = H. 



By lemma 15.41 we obtain 



(5i 



(5.9) 
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Then, 



< P< 





Jo 


■■) 




Jo 





(/, - 9.) dW, 



^1 \^fjj\ > 2 



z 

> — 

H 4 



< p. 



2 J " \ ''H 4 
As r/ is a standard Gaussian random variable, we can write for all z > 2, 

z\ 12 '•+°° 
2/ V vr 



z/2 
J z/2 



2 -22/8 



We can also express the second term on the right-hand side of ( IS.lOp as 



P5 U- >-r 

^ ^^-^H 4 



+00 



z 



1=1 ^ 

+00 / 9 l — l 



1=1 

+00 



= E (l^^l > 1) ^s{rH = I) 
1=1 

1=1 

So for any z > 2, flS.lOp implies 



> z] <2 



□ 
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Lemma 5.6. iHelland I . \l98i pp. 80-82) 

Let ('Ufc,n)i<fc<n be a "martingale difference" defined on a probability space {Q,J^,V) and 
filtrations {J^k,n,k E N} of J^, n E W such that Uk,n is J^k,n-'mGasurable. Assume that the 
following two conditions are satisfied: 



(1) 
(2) 



k=i 



Y] E(u^^„l(K„|>e)|J'fc_i,n) y 0, for all e > 0, 

n 

EKnl-^fc-l,n) 1- 



fe=i 



Then, 



^A/'(0,1). 

k=l 

6 Numerical simulations 

We illustrate the obtained results by the following simulation which is established using 
Scilab. 

The purpose is to estimate, at a given point Zq, the function S defined over [0; 1] by 
S{x) = \x — zq\^ . We check that such a function belongs to 'H^'^^Zq, K,e) when K > 1. 
The values of zq and /3 are arbitrary, which permit the user to name his choice. As an 
example, take zq = l/\/2. Then /5* = 0.6 is a lower regularity value and /3* = 0.8 is the 
higher regularity value. 

We simulated n data for the function S{x) = |a; — ZqI'^ for P = 0.7. We obtained 
an estimation in constructing the estimator Sn defined in f l2.7p with the procedure of 
Lepskii which gives us the optimal window for the index k defined in f l2.6p . 

Numerical results approximate the asymptotic risk of a sequential estimator defined in 
(12. 7p used due to the calculation of an expectation (it performs an average for M = 15000 
simulations) and the finite number of observations n. Here we calculate for the sequential 

1 

estimator the quantity R„ = — |5'^'^-'(zo) — S{zo)\. 



k=l 



By varying the number of observations n, we obtain different risks listed in the follow- 
ing table: 



When taking /3 = /3* 



n 


100 


1000 


5000 


10000 




0.284 


0.154 


0.101 


0.087 


1, we obtain 


n 


100 


1000 


5000 


10000 




0.201 


0.097 


0.058 


0.047 
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